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Abstract 


A team of instructional designers, educators, and the School of Liberal Arts (SLA) academic program 
coordinator from a nonprofit online college, collaborated on producing short voice-over videos with 
interactive elements that address the most common technology frustrations of beginning students. These 
videos were inserted into the “Start Here” page of the pilot study courses. Courses were selected that had 
a higher than average withdrawal rate and that also tended to have a broad grade distribution for students 
who completed the course. The videos with voice-over covered basic navigation, such as posting to a 
discussion board, submitting an assignment to a drop box, reviewing a grading rubric in the grade book, 
and opening a graded copy of an assignment to view instructor feedback. The orientation element 
contained interactive tools that allowed students to check their learning of the skills covered in the videos. 
The orientation videos were available throughout the 8 weeks of the course. After viewing the videos, the 
students were asked to complete a short survey, which included both quantitative and qualitative 
feedback. The responses to the survey were extremely positive, ranging from 84.48 to 90.67 on a 100- 
point scale. The team also examined course withdrawal rates and the grade distribution before and after 
the insertion of the videos. The data for almost all of the pilot study courses showed a drop in withdrawals 
after the use of the video orientation element. Several courses showed a significant drop in withdrawals. 
The examination of the pre- and postintervention grade distribution showed an improvement in grades for 
almost all of the pilot courses. This improvement in grade distribution was statistically significant in some 
courses. 
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Introduction 

Online education is growing quickly across all colleges and universities (Wladis, Wladis & 
Hachey, 2014). With this enrollment increase come new challenges pertaining to retention and 
persistence. Degree completion is progressively becoming a larger issue, particularly within online 
programs. According to the National Student Clearinghouse Research Center, completion rates for 
students who are older or part-time have increased. According to their 2008 data, 42.1% of enrolled adult 
students completed their degree (2014). For the purpose of this study, we are defining adult students as 
those who are 24 years of age and older. This age range is the reporting measure used for adults by the 
National Center for Educational Statistics (2014) and the National Student Clearinghouse Research 
Center (2014). It is also reflective of this study’s population. 

The purpose of this study is to address retention and persistence concerns in adult online 
education. The authors propose that if instructors embed an orientation within the class itself, students are 
more likely to complete the course. Park and Choi enumerated factors influencing adult student 
persistence in online classes, listing the learners’ skills as an area that has an effect on the students’ 
success but is outside of the control of the instructional design (2009). Given this assertion by Park and 
Choi, the authors felt that instructional design and orientations can be developed for the purpose of 
augmenting these skills. Such augmentation could lead to the students’ overall success and completion of 
the course. 

This study was designed to determine if providing just-in-time tutorials and course orientation 
would improve course completion rates. 

Theoretical Significance of This Study 

When discussing adult retention, it is beneficial to review what “adult learner” really means in the 
United States (since this study was conducted at an online college in the United States, it makes sense to 
work within this context). The word for adult education is andragogy; the term has been around for a long 
time. It was coined by Alexander Kapp in 1833 and meant “education in a man’s age” (Reischmann, 
2004). Since Kapp’s time, many scholars have written about adult education. However, in the United 
States, Malcolm Knowles is attributed with creating both a theoretical perspective and an operational 
framework for approaching adult education. Knowles’ theory has become foundational for those working 
in the adult learning field (Reischmann, 2004). Knowles wrote that it is erroneous to think of adults in the 
same way we think about children and adolescents when it comes to learning (1973). Adults are more 
mature and live complicated lives in which every moment is accounted for. Knowing that adults’ lives are 
complex, Knowles created a set of assumptions that are meant to guide how educators tailor their classes 
to better serve adult students. These assumptions are the following: (1) adults are more self-directed; (2) 
adults bring rich experiences; (3) adults are more ready to learn; (4) adults want to apply what they learn 
directly to their world; (5) adults learn through problem solving; and (6) adults want to know why they 
are learning what they are learning (Knowles, 1980; Glancy & Isenberg, 2013; Chan, 2010). Out of this 
list of assumptions, the one most relevant to this study is adults’ self-directedness. 

Online education requires students to be more self-directed (Jones, A., 2013). Online courses are 
more flexible and malleable to students’ individual situations (Park & Choi, 2009) and require students to 
take more initiative in their learning processes. While this malleability is a positive attribute of online 
courses, it can also be a hindrance. Despite the flexibility, the amount of work can be overwhelming, 
creating cognitive overload (Jones, A., 2013). The courses often are text heavy, requiring an intensive 
amount of reading and writing. An instructor who meets students face-to-face during the week to assist in 
keeping the learning on track is absent. These students must be completely reliant upon themselves to 
complete assignments and discussions in a timely fashion. Students also have to be prepared to make a 
large time commitment alongside their other everyday expectations. Candy (1991) writes that faculty 
must orient, guide, and foster this independent learning among adult learners, whether online or face-to- 
face. One way to set adults up for success in an online environment is to ensure that they feel comfortable 
in that setting. 
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A report produced by Complete College America gives suggestions on ways to increase student 
course and degree completion. They write that one way to improve retention is by providing integrated 
support in “college-level gateway” courses (2014). Excelsior College, the setting for this study, is a 
degree-completion college with a very low residency requirement. Many students take as few as one or 
two courses with Excelsior before graduating. Thus, for this study, “gateway courses” can be defined as 
the first course taken by a student at the college. It is essential that students learn how to orient themselves 
to the course upon college entry. 

Orientations have the ability to assist students in understanding how to use the learning 
management system (LMS), available services, and course expectations. A rural community college 
created a mandatory online orientation program for its students and found that students were better 
prepared for this educational experience as a consequence (Jones, K. R., 2013). Orientations, however, 
are not the panacea for increasing retention. Another factor contributing to student success, according to 
the Complete College America report, is the use of a system of intrusive just-in-time advising (2014). 
While the report is geared to young adults, this factor can be extrapolated to adult online students. 
Students who are returning to college with a plethora of credits need support that is tailored to their needs 
at that moment. Instead of taking the voluntary online orientation, students tend to jump into their classes 
without knowing much about what to expect or how to navigate in the LMS. Thus, looking at a pathway 
for adult students to succeed requires just-in-time support inside the LMS that is quick, easy to use, and 
interactive. Hence, not only can an orientation serve to get students prepared for how to participate in the 
course, the orientation can also provide just-in-time tutorials to assist students when they are stuck on 
how to work within the course. 

Study Background 

Excelsior College is a not-for-profit online institution serving adult students with an open 
admissions policy. The mission of the college focuses on adult learners and those who have been 
historically underrepresented in higher education. The School of Liberal Arts (SLA) allows the most 
transfer credits of any school within Excelsior College. Students come to the SLA when they are near 
degree completion; quite often they are also engaging in their first online learning experience. The focus 
of this study was to see if incorporating an orientation video with interactive elements into the opening 
page of courses could improve student success in these online learning experiences. Online learning can 
create additional challenges for students beyond those that they might encounter in face-to-face 
classrooms, especially for this student population, who have often been less than successful in traditional 
colleges. As an open-access college focused on adult returning students, often from underserved 
populations, retention and persistence rates are a primary concern for the institution. The five schools of 
the college have been challenged to submit proposals for projects that will improve retention and 
persistence and, thus, graduation rates for their students. Small internal grants were awarded to cover the 
cost of these pilot interventions. The authors, who are from the SLA, proposed to create simple 
orientation videos with narration that included interactive elements. The proposal was accepted, and the 
authors were awarded a small grant in October 2012. The concept was to create short video tutorials that 
walked students through the most basic procedures they needed to master to be successful in their course. 
Each video was less than four minutes in length. A menu was provided so that students could skip around 
or repeat any element. Topics covered include the following: 

• How to get started in the course 

• Module navigation 

• Posting to discussions 

• Submitting assignments 

• Locating your grades, instructor feedback, and grading rubrics 
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Figure 1. The welcome to the course video. 



▼ 1. BIO*212 Course Orientation 

1.1. Welcome 

1.2. Course Information Video 

1.3. Quick Video Review 

1.4. Module Navigation Video 

1.5. Hotspot Activity 1: 
Homepage 

1.6. Hotspot Activity 2: 
Breadcrumbs 

1.7. Hotspot Activity 3: Page 
Navigation 

1.8. Discussions & Assignments 
Video 

1.9. Characteristics of Discussions 
v. Assignments 

1.10. My Grades Video 

1.11. Wrap Up & Summary 


Welcome to 1310*21-2-: 
Microbiology! 


Please participate in this short activity 
to learn how to best interact with your 
course and. understand what 
is expected of you. 


4 ► | O < PREV NEXT > 

The orientation begins with a welcome to the course video as shown in Figure 1, which invites 
the student to feel comfortable with the course environment. The entire video orientation and navigation 
product were created using the two software packages: Articulate Storyline and Camtasia Studio. Viewing 
the videos and participating in the interactives takes no more than 10 minutes. These orientations were 
inserted into selected courses that met the following three criteria: (1) high enrollments, (2) often one of 
the first courses taken by new students, and (3) higher than average withdrawals. 

Figure 2. One of the interactive elements for self-assessment. 



1.4. Module Navigation Video 

1.5. Hotspot Activity 1: 
Homepage 

1.6. Hotspot Activity 2: 
Breadcrumbs 

1.7. Hotspot Activity 3: Page 
Navigation 

1.8. Discussions & Assignments 
Video 

1.9. Characteristics of Discussions 
v. Assignments 

1.10. My Grades Video 

1.11. Wrap Up & Summary 




Quick Review 

Even though that video was 
only 2 1/2 minutes in length it 
covered a lot of really important 
things about your course. Let's 
quickly review the information 
you learned during the video. 

Click each sticky note to the 
right to review the items 
covered. 


4 ► 
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The videos and interactive elements work in concert with one another. The inclusion of brief 
interactive self-assessments inserted directly after the short video allows the students to check their 
understanding of the information presented. Figure 2 is an example of such an activity. The mixture of 
videos and interactive elements was created for the purpose of building student success in the course. 

Method 

Student success was evaluated using both a qualitative and a quantitative measure. The end of the 
orientation video section contained a short six-question survey designed to evaluate students’ perceptions 
of the video orientation. Additionally, the withdrawal rates and grade distributions for each course were 
examined before and after the insertion of the video orientation. 

The survey was designed to allow the students to give a “grade” for their response to each of the 
stem items from “0 (strongly disagree )” to “100 {strongly agree)." It was felt that a 100-point scale would 
be a familiar format for students—similar to a grade—and would have intrinsic meaning for them. The 
final two items of the survey allowed for an open-ended response. Students were not required to view the 
video orientation, nor were they required to take the survey. The survey was constructed and responses 
were compiled using the Qualtrics survey system. 

The survey consisted of the following six items: 

1. Yes, the video was very clear and easy to understand. 

2. Yes, the videos were informative. 

3. Yes, the interactive elements improved my ability to navigate inside the course. 

4. Yes, as a result of this orientation, I feel more knowledgeable about how to work 
inside my course. 

5. What were the most helpful parts of the orientation? 

6. What areas could be improved to better address your questions or concerns about 
navigating inside your course? 

The video orientation first appeared in the 2013 Summer 1 term in HUM300 and B10212. In May 
2014, it was adapted for three additional courses and added to H1S101, SOC101, and ENG101. ENG101 
is an introductory composition course that runs for 15 weeks. The other courses all run for 8 weeks each 
term. As stated previously, these are courses that run many sections each term and are often taken early in 
a students’ enrollment with Excelsior College. 

The courses selected had a stable population of instructors and had not been revised or 
substantially changed in the year prior to the video, during which time the video orientation was included. 
The withdrawal rates for each course were examined for the one-year period before the insertion of the 
video orientation. These rates were then compared to the withdrawal rates during the time when the video 
orientation was in the course. The grade distributions in each course were also examined for the year prior 
to the video. This data was then compared to the grade distributions during the period when the video 
orientation was present in the course. This data was collected using the data management system of the 
college where information for each section of each course is accumulated. The college currently uses the 
QlikView Business Intelligence system. This software allows specific date-range selection along with the 
ability to track specific courses for withdrawals and grade distributions. 

The response rate varied somewhat for each question and for the additional comments on 
Questions 5 and 6, which were optional. The response numbers for individual survey prompts 1 through 4 
appear in Table 1. The number of responses for the first four prompts in the survey ranged from 802 to 
817. For the first four Likert-scale prompts, the average A was 810 responses. 

Survey Results 

The results from the survey responses are presented in Table 1. The results for all of the survey 
responses from all of the courses were combined for this report. 
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Table 1. Responses to the survey for the four Likert-scale prompts. 


Survey prompt 

Average 

score 

Standard 

deviation 

Number of 
responses to 
this prompt 

1. Yes, the video was clear and easy to understand. 

90.67 

20.33 

817 

2. Yes, the videos were informative. 

90.57 

20.30 

802 

3. Yes, the interactive elements improved my ability to 
navigate inside the course. 

84.38 

24.68 

812 

4. Yes, as a result of this orientation I feel more 
knowledgeable about how to work inside my course. 

86.18 

23.92 

809 


The final two survey items were questions and allowed for open-ended responses. The responses 
were coded, and we then looked for common elements in the parts of the orientation identified. The 
number of responses for each is given below. 


Question 5, “What were the most helpful parts of the orientation?” received 100 responses. 
Twenty percent of these students identified the navigation instruction as the most helpful element. 
Additionally, 29% identified the interactive topic review exercises as most helpful. The rest of the 
responses were varied. Examples of comments received from Question 5 included the following: 

• The use of the cartoon characters was fun 

• 1 learned that the syllabus and all the course information is always at my fingertips whenever 
1 get confused. This is my first online course 

• The exercises 

• How to locate the discussion and assignment links in the course 

• The way the video moved around and used visual teaching 

• The interactive activities 

Question 6, “What areas could be improved to better address your questions or concerns about 
navigating inside your course?” received 95 responses. A total of 55.8% of the students specifically stated 
that nothing could be improved, 17% identified difficulty with playing the videos or getting the 
interactive to work, and the rest had specific design suggestions. A sample of comments received for 
Question 6 included the following: 

• 1 wish it said “assignments” instead of “home page and module” 

• It doesn’t explain what breadcrumbs are 

• Augment the narration and pointer with highlighted field 
Withdrawal Rate Results 

After the insertion of the orientation video, four of the five courses involved in the study showed 
a reduction in the percentage of students who withdrew compared to the previous year. Two of these four 
courses showed a statistically significant reduction in these withdrawal percentages. One course 
(SOC101) showed an increase in withdrawals, but this number was not found to be statistically 
significant. Table 2 shows the results of the withdrawal rate comparison. The number of students in each 
course before and after also appears in parentheses following the percentage change in Table 2. 


Table 2. Withdrawal rates before and after the video orientation was introduced into the course. 


Course 

Before video 
orientation withdrawal 
rate (A) 

After video 
orientation, 
withdrawal rate (/V) 

Change after video 
orientation 

BI0212 

45.3% (658) 

32.7% (1,433) 

-12.6%* 

HUM300 

30.5% (387) 

23.7% (716) 

-6.8%** 

ENG101 

21.8% (2,263) 

19.6% (387) 

-2.2% 

HIS101 

25.9% (475) 

24.9% (94) 

-1.4% 

SOC101 

20.3% (541) 

22.3% (233) 

+2.0% 


*p = .0001. ** p = .0074 


Online Learning - Volume 19 Issue 4 - September 2015 

























Innovative Orientation Leads to Improved Success in Online Courses 


The grade distributions for these courses were also examined before and after the introduction of 
the orientation video. Excelsior College awards only whole letter grades. The passing grades (A-C) were 
looked at as one data point, and the D and F grades were also separately examined for each course. There 
were statistically significant increases in the percentage of A-C grades awarded in three of the courses: 
H1S101, B10212, and HUM300. SOC101 experienced a statistically significant decrease in the 
percentage of A-C grades. No other changes were significant. The following paragraphs provide further 
detail. 

HIS 101 showed an improvement in the percentage of A-C grades awarded after the introduction 
of the video orientation. This group showed a 15.4% increase, which is a statistically significant change 
(p = .0019). In HIS101 there was a slight decrease (2.1%) in students receiving a grade of D, which was 
not statistically significant. There was an increase (3.1%) in the percentage of F grades awarded, which 
was not statistically significant. 

B10212 showed an improvement in the percentage of A-C grades awarded after the introduction 
of the video orientation. This group showed an 8.4% increase, which is statistically significant (p = 
.0002). There was an increase in the percentage of students receiving either a D grade (2.1%) or an F 
grade (0.9%), neither of which was statistically significant. 

HUM300 showed an improvement in the A-C grades awarded after the introduction of the video 
orientation. This group showed an 8.0% increase, which is a statistically significant change (p = .0032). In 
HUM300 there was a slight increase (2.1%) in the percentage of students receiving a grade of D, which 
was not statistically significant. There was a decrease (2.2%) in the percentage of F grades awarded, 
which was not statistically significant. 

ENG101 showed a slight improvement in the A-C grades awarded after the introduction of the 
video orientation. This group showed a 3.5% increase, which is not a statistically significant change. In 
ENG101 there was a slight decrease (0.5%) in the percentage of students receiving a grade of D, which 
was not statistically significant. There was also a slight decrease (0.3%) in the percentage of F grades 
awarded, which was not statistically significant. 

SOC101 showed a slight decrease in the A-C grades awarded after the introduction of the video 
orientation. This group showed a 4.8% decrease, which was a statistically significant change (p = .0032). 
In this course, there was a slight increase (0.6%) in the percentage of students receiving a grade of D, 
which was not statistically significant. There was a slight increase (0.9%) in the percentage of F grades 
awarded, which was not statistically significant. 

Discussion 

The results seem to indicate that a video orientation with interactive elements can improve 
desired outcomes when inserted into online courses that are frequented by novice online learners, have 
high withdrawal rates, and have broad grade distributions. These types of video orientations are also 
favorably received by students and seen as informative and helpfiil by survey participants. This was a 
small pilot study, and more data from a wider variety of courses needs to be collected before any 
definitive conclusions can be drawn. The statistically significant improvement in grades and reduction in 
withdrawals seen in some of the courses is a hopeful indicator that embedding these simple, economical 
orientation videos into online courses can improve student success under certain conditions. 

It should be noted that almost all of the statistically significant positive change was experienced 
in the mid- to upper-level courses (BI0212 and HUM300). One possible reason could be the much larger 
after-video N values for BI0212 and HUM300 versus the other courses. Or perhaps the student 
population in introductory courses is markedly different than that of these other high-enrollment classes. 
More research is needed to test such hypotheses. We will be continuing to collect data and expand this 
video orientation to additional courses in the next year. 

The results from the SOC101 course are troubling. The researchers are attempting to track down 
any unique elements that impacted this course during the study period. 
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There are several limitations to this study. The number of courses selected was small, and the 
courses were not randomly selected, but targeted first-time online course takers. The results provided may 
show a correlation between the use of the orientation video and improvements in student success, but 
directly linking these two events is problematic. An additional limitation is the use of only one college. In 
many ways Excelsior College attracts a unique student population. Most have accumulated credits at 
other institutions and through military and civilian credit-bearing activities outside the classroom. This 
preparation and experience makes the Excelsior student body dissimilar from a traditional student 
population. 


Conclusion 

Much has been written about how to engage in “entering services” (Chickering & Reisser, 1993). 
One facet of this engagement is the orientation. To date, orientations are delivered through a variety of 
methods, from face-to-face to blended to online. Most online institutions have an orientation course that 
students can take voluntarily. However, many busy adult students don’t engage in or complete it. They 
really need another form of what Chickering and Reisser (1993) call “initial assimilation.” Students need 
to adapt to the online environment in order to be successful. The more familiar the course tools and 
navigation are, the easier it is for the students to reach completion, as they don’t have to think about how 
to get around in the LMS; this correlation is evidenced in the improved completion rate in four of the five 
courses in this study. This relationship between familiarity with course tools and navigation and student 
completion is especially true for short and intensive courses done in 4 to 8 weeks. 

This article starts a conversation as to what methods work best when orienting students to the 
online environment. In this study, the authors found that using just-in-time tutorials to assist with both 
initial assimilation and practical assistance worked in most but not all cases, which raises questions about 
what other factors come into play with regard to online adult student success. Along with creating 
welcoming orientations with just-in-time tutorials, what else can be created for the purpose of improving 
student success? Might there be changes in the design, navigation, or instructions that impact student 
completion? Or might there be changes within the orientation itself, such as changing how videos open so 
that these can play in any browser or on any computer, that have a positive effect? Of course, there is 
always the possibility that because these are adult students who live full lives, sometimes living life can 
get in the way of completion. 

Nonetheless, the authors found that student orientations are important for familiarizing learners 
with the educational environment and lead to improved rates of course completion in most cases. The 
opinion of the authors is that online learning has the potential to open up new methods to encourage 
student educational engagement both for online and face-to-face institutions. The virtual world allows us 
to explore new and creative ways to ensure that students can be successful. 
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